{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Making COVID infections, deaths, and infections/death ratio predictions on Metaculus more consistent with each other\n",
    "- More broadly: a workflow that connects judgmental and model-based forecasting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "%pip install --quiet poetry  # Fixes https://github.com/python-poetry/poetry/issues/532\n",
    "%pip install --quiet git+https://github.com/oughtinc/ergo.git@7f88222f5f7a2e552eb1750d43b6c7924d2f0361"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(action=\"ignore\", category=FutureWarning)\n",
    "warnings.filterwarnings(module=\"plotnine\", action=\"ignore\")\n",
    "warnings.filterwarnings(module=\"jax\", action=\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ergo\n",
    "import seaborn\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metaculus = ergo.Metaculus(username=\"oughtpublic\", password=\"123456\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Look at questions on Metaculus\n",
    "* [Total COVID infections before 2021](https://www.metaculus.com/questions/3529/how-many-infections-of-covid-19-will-be-estimated-to-have-occurred-before-2021-50k-1b-range/)\n",
    "* [Total COVID deaths before 2021](https://www.metaculus.com/questions/3530/how-many-people-will-die-as-a-result-of-the-2019-novel-coronavirus-covid-19-before-2021/)\n",
    "* [Chance of dying of COVID if you get it](https://www.metaculus.com/questions/3755/what-will-be-the-ratio-of-fatalities-to-total-estimated-infections-for-covid-19-by-the-end-of-2020/)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load questions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "q_infections = metaculus.get_question(3529, name=\"infections\")\n",
    "q_deaths = metaculus.get_question(3530, name=\"deaths\")\n",
    "q_ratio = metaculus.get_question(3755, name=\"ratio\")\n",
    "\n",
    "questions = [q_infections, q_deaths, q_ratio]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show community estimate for each question"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[question.show_community_prediction() for question in questions];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model deaths based on the infection and ratio community estimates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deaths = infections * deaths/infection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "def deaths_from_infections():\n",
    "  infections = q_infections.sample_community()\n",
    "  ratio = q_ratio.sample_community()\n",
    "  deaths = infections * ratio\n",
    "  ergo.tag(deaths, \"deaths\")\n",
    "  return deaths\n",
    "\n",
    "samples = ergo.run(deaths_from_infections, num_samples=5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How does our model prediction compare to the community prediction?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "q_deaths.show_prediction(samples[\"deaths\"], show_community=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combine our model with the community prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mostly defer to the community's predictions on the deaths question, but update a bit towards the model that's based on infections * deaths/infection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "def deaths_adjusted():\n",
    "  if ergo.flip(.66):\n",
    "    deaths = q_deaths.sample_community()\n",
    "  else:\n",
    "    deaths = deaths_from_infections()\n",
    "  ergo.tag(deaths, \"adjusted_deaths\")\n",
    "\n",
    "samples = ergo.run(deaths_adjusted, num_samples=5000)\n",
    "\n",
    "adjusted_samples = samples[\"adjusted_deaths\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "q_deaths.show_prediction(adjusted_samples, show_community=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submit new distribution on deaths to Metaculus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "q_deaths.submit_from_samples(adjusted_samples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Apply the same idea to estimating infections from deaths"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
